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(54) RALLY GAME SYSTEM USING VIEW LABEL 

(57)Abstract: 

PROBLEM TO BE SOLVED: To present computer- 
derived topographic information and an image showing 
real view to each user while matching the topographic 
information with each part of the image and to make an 
automobile rally game possible in a real city. 
SOLUTION: This system transmits position mformation 
(with time) about the automobiles of users and 
automobile information from a terminal 10 to a center 20 
at regular time intervals. A game information control part 
25 controls the position infonmation about each 
automobile and the automobile information, and delivers 
the position information about the users and order 

information to a label information originating part 22. The Pil ;^.™iv ,irv^^^^)^i^^->'^^^^'^--'^''^'^-^ , v. i = 
label infonnation originating part 22 originates rally order ^^^^-^^^^^^^^^-^^-^ 
label information such that rally order information corresponds to an automobile area in an 




http://vwwl9.ipdlanpit.go.jp/PAl/re$uI^^ 4/15/2009 

PAGE 21/51 ' RCVD AT S/13/2009 11:27:09 AM [Eastern DayiigM 



MAY. 13. 2009J0:28PM NAVTEQ CORP NO. 942 paKP. 22> 



image of the view. The rally order label information is delivered to the label information output 
part 17 of the terminal 10 and displayed on the windshield of the automobile of each user. 
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* NOTICES* 

JPO a^^d INPIT afe not responsible for any 
damages catised by th& use of this translation. 

LThis document has been translated by computer So the translation may not reflect the 
original precisely. 

2-**** shows the word which can not be translated. 
3.ln the drawings, any words are not translated. 



DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Reld of the lnvention]This Invention carries out the superimposed display of the geographical 
information about each subregion In the picture to an Image display device to the picture which 
the user photoed using landscape image Input devices, such as a camera, or relates to the 
device which carries out an audio assist etc. and is taught to a user. 
[0002] 

[Description of the Prior Art]ConventionaIly, there were various navigation systems as a 
system which teaches a user the geographical information about the circumference In which a 
user is present 

fQ0Q31 Drawing 12 I s a lineblock diagram of the navigation device Indicated by JP,8-273000A 
The position updating part 71 which ymW update the position of vehicles with reference to road 
map data if this device inputs the position data and motion data of vehicles, The data 
generating part 72 for a display which generates the road data for a display, and the 
background data for a display based on map data etc., The three-dimensional dynamio-image- 
data preparing part 73 which creates three-dimensional dynamic image data based on these 
data for a display, It has the storage parts store 74, and when the user of a navigation device 
sets up a priori a running path including the destination and a course place, It has a function in 
which a course can be set up, looking at the real dne mode display screen along the road 
which actually exists instead of a map screen. 

[0004]According to this device, the user can see the cine mode display (for example, drawing 
13) in alignment with that course, when running in accordance with the course which actually 
exists. 
[0005] 

[Problem(s) to be Solved by the lnventlon]However, when using the device, and human being 
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matches actual scenery and the geographical infonnation in the world of a computer eventually 
with the naked eye, It must be recognized what the thing in actual scenery is. That is, based on 
the sign in the map with which the cine mode display of what the actual building and road in a 
user's view, and a mountain are was canied out, etc.. a naked eye is made reliance, human 
being's brain is used unconsciously, and human being has to work matching and has to 
understand. If the map in a computer and the actual scene were compared in the street comer, 
the direction has been grasped, or the mark was found and it gazed at the direction, after 
understanding the feature of the building which exists in the direction, the map was seen 
again, and he understands what the building is. 

[0006]For this reason, the time and effort which compares the map and real landscape on a 
computer repeatedly, and is matched at human beings has the problem of not being omissible. 
A real landscape is hard to see and cannot take correspondence easily especially in 
semidarkness, the night, etc. 

[0007]There is a feult that the automobile rally game in the Inside of a personal computer has 
few senses of reality, and they are unsatlsfiactory. 

[0008]The purpose of this invention is to provide the scene (abet use type rally game system 
which can do a rally game in the actual street while matching each portion in the geographical 
infonnation on a computer, and the picture (it is hereafter called a landscape image.) of a real 
landscape and teaching a user. 
[0009] 

[Means for Solving the ProblemlThis invention creates map data on a computer beforehand as 
three-dimensional data, A position in case a picture (it is henceforth called a landscape image 
in order to distinguish from a CG Image) is inputted, an angle of a camera, a focal distance, 
and image size are acquired at the time of photography. Computer graphics at the time of 
viewing from a position at the time of real landscape photography, an angle of a camera, and a 
focal distance in three-dimensional map space on a computer. (It is hereafter referred to as 
CG.) Geographical infonnation within a picture is acquired and matching is realized by carrying 
out the superimposed display of the geographical information to a landscape Image which is a 
real landscape. This geographical iriformatlon is names, such as a structure, or attribution 
information of those in a picture, and attribution infonmation means infonmatlon about all 
attributes (for example, an outline, a color, etc.) about that stmcture. Suppose that the word 
"stmcture" is used in the sense of all data that has a certain geographical stoictures in the map 
DB also including natural geographical features, such as a mountain, a river, and the sea, in 
addition to an artificial structure in this specification- In acquisition of geographical information, 
it asks for a landscape image based on a camera position, a camera angle, a focal distance, 
and image size, and asks for a structure of a multiple image. A position (a grant position is 
called hereafter) of a landscape image in which the structure should be reflected is searched 
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for, and the superimposed display of a name or attribution information of a structure is carried 
out 

[OOlO]ln order to raise further accuracy of matching with a structure in a landscape image, and 
a structure in a CG image, a structure previously gained to each subregion of a landscape 
image is matched with pattern matching. A CG image is created based on a gained stmcture, 
and it asks for a structure which matched subregion in a CG image with pattern matching to 
said subregion of a landscape image, and became a basis of matched subregion. 
[001 1]Here, an example of a method of creating a CG image is described. The three- 
dfmensionaf map DB is accessed based on a camera position, a camera angle, a focal 
distance, and image size which were acquired previously, and it asks for view space in three- 
dimensional map space. It asks for a structure in view space, and three-dimensional projection 
conversion of the solid data of each structure is carried out to this surface of projection by 
using a camera screen as a suri'ace of projecBoa It hides in other structures among line data 
which furthermore constitutes a projection form of each structure, and the hidden line removal 
of the hidden-line data is carried out using techniques, such as a normal vector method. A 
hidden line removal is carried out and area division of the CG image is earned out based on 
the remaining line data. Since the three-dimensional map DB is used, a name of a structure 
which serves as a basis of the field for every field can be matched. 

[0012]Then, a stmcture name of subregion of a CG image matched with each subregion of a 
landscape image by. pattern matching is extracted. Three-dimensional projection conversion of 
the position coordinate of a structure in inside of three-dimensional map space Is carried out to 
a previous suriace of projection, and a position coordinate of a real landscape picture which 
should superimpose an extracted structure name is searched for. Label information is created- 
from a position coordinate of a real landscape picture which should superimpose an extracted 
structure name. A structure name is superimposed on a landscape image which is a real 
landscape based on label Information, and it displays on vision apparatus. 
[0013]ln this invention, in order to make an automobile rally game in the still more neariy actual 
street, the following composition is taken. 

I0014]A communication control means sends position information (with time) and car 
information about a user to a center for every fixed time from a tenminal. Here, having called it 
car Information consists of attribution information (for example, a registration number of a car, 
the vehicle name / model / color of a car, using frequency of a transceiver of automobile 
loading, etc.) about a car which each user drives. A communication control means by the side 
of a center passes position information and car information about a user to a game information 
management tool via a center control means. In a game information management tool, each 
user's position information and car information are managed, precedence information In a rally 
of each car computed flrom position information about a car which a user who parti'cipates in a 
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game drives, and position Infonnation and map information is managed, and position 
information and precedence information are passed to a center control means. In a center 
control means, car information is passed to a user's position information and a precedence 
Information pan at a label information preparing means. In a label information preparing 
means, based on a user's position information and car information under intervention, When a 
car which exists in a landscape image is recognized and a registration number of the car has 
been recognized from a number plate of the car» If it can presume that the registration number 
is a registration number of a car v/hich a certain user drives from a point of position information 
and car information, rally ranking label information which made precedence information of a 
rally correspond to a field of a car in a landscape Image will be created, and, in addition to the 
usual label information, a center control means will be passed. Here, it becomes rally ranking 
label Information from attribution information including a name of an automobile slack structure 
which a user under intervention operates, and precedence infbmiation, and its position 
information. Via a communication control part over a label information outputting part, rally 
ranking label information superimposes a ranking number on a picture of a car, and rs 
displayed on a vWndshield of a user's car. 

[001 5]A scene label use type rally game system of this invention. Consist of two or more 
terminals and centers which were carried in a car, respectively, and said each terminal, An 
image acquiring means which acquires a picture, and a position information acquiring means 
which acquires a camera position at tiie time of image acquisition, A camera attribution 
information acquisition means which acquires a camera angle, a focal distance, and image 
size at the time of image acquisition, An image processing means which divides an acquired 
picture into two or more subregions, and information about area division, said camera position, 
said camera angle, said focal distance and said image size of said picture via a 
communications network, A communication control means which transmits to a center, 
transmits to a center car information which is position information with time on a car of tiie 
terminal concerned, and the attribution information of a car via a communications network for 
every fixed time, and receives label Information and rally ranking label information from said 
center, A name of a structure in said label information or its attribution information is 
superimposed on a position conresponding to a grant position in a picture, Have a label 
information output means which displays a picture on which it was superimposed on >rtsion 
apparatus, and displays said rally ranking label information on a windshield of a car, and a 
terminal-control means to control said each means, and said center, Infomiation about area 
division of said picture, said camera position, said camera angle, said focal distance, said 
image size, position information witii time on each terminal, and car information are received 
from said each terminal via said communications network, A communication control means 
which ti-ansmits said label information and said rally ranking label infonmation to said terminal, 
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A map information management tool which gains a structure which manages map Information, 
asks for view space in map Information space based on a camera position, a camera angle, a 
focal distance, and image size which were received, and exists all over the view space, A 
game information management tool which manages position information with time and car 
Information of each terminal, and manages precedence information in a rally of each car which 
a user who participates in an automobile rally game drives, Said label Information which 
matches said gained structure to said subreglon of said picture, and includes a name or 
atbibution information, and a grant position of said structure which were matched Is created, 
Based on a user's position information and car infomiation under rally game intervention. 
When a registration number of the car has been recognized from a number plate of the car and 
the registration number can presume a car which exists in a landscape image to be a 
registration number of a car which a certain user drives from a point of position information and 
car information. It has a label information preparing means which creates rally ranking label 
information which made precedence Information of a rally game correspond to a field of a car 
in a Iandsca|>e image, and a center control means which conti^ols said each means, 
[0016]According to the embodiment of this invention, a label information preparing means, A 
CG image which is a computer graphics image is created based on a gained sbucture, 
Subregion in said CG image is matched with pattem matching to said subregion of said 
pictijre, It asks for a structure of matched subregion, and label information including a name or 
attribution Information, and a grant position of tiie structure is created. 
[OOlTJAccording to other embodiments of tills invention, a label Information preparing means, 
Canry out tiiree-dimenslonal projection conversion of the gained stiuctijre to a camera screen, 
eliminate a structure which does not appear from a viewpoint, and a CG image is created, A 
border line of subregion in a CG image divides a CG image into subregion. It asks for a 
structure which matched said subregion of said picture, and said subregion of said CG image 
with pattern matching, matched to subregion of a picture, and became a basis of subregion of 
a CG image, and label information including a name or attribution infonnation, and a grant 
position of the stmcture is created. 
[0018] 

[Embodiment of the lnvenh'on]Next, the embodiment of ttiis Invention is described vwtti 
reference to drawings. 

[0019]R6ference of drawing 1 w ill constitute the scene label use type rally game system of one 
embodiment of this Invention from the terminal 10. the center 20, and the communications 
networic 30 which were carried in the car. Here, two or more terminals 1 0 exist actually, 
altiiough the deer graphic display even of tiie **** top is not carried out. 
[00201The position information acquisition part 12 which acquires the Image acquiring part 11 
from which the terminal 10 acquires a picture, and the camera positi'on at the time of image 
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acquisition, and the camera attribution information acquisition part 13 which acquires tha 
camera angle, the focal distance, and image size at the time of image acquisition. Jhe image 
processing portion 14 which divides the acquired picture into two or more subreglons, and the 
Information and camera position about area division, the camera angle, the focal distance and 
image size of said picture are transmitted to the center 20 via a communications network, the 
attribution information (the registration number of a car.) about the position information with 
time on a car and the car 10 of the terminal 10 The car information which consists of using 
frequency of the transceiver of a vehicle name / model / color, and automobile loading, etc. is 
transmitted to the center 20 via the communications network 30 for every fixed time. The 
communication control part 15 which receives label infomnation and rally ranking label 
information from the center 20, The label information outputting part 17 which superimposes 
the name of the structure in label Infbnnation, or its attribution information on the position 
conresponding to the grant position in a picture, and displays the picture on which it was 
superimposed on vision apparatus, and displays rally ranking label information on the 
windshield of a car, it comprises the terminal control section 16 which controls each part 1 1- 
15, and 17, 

[0021]The center 20 receives the infonnation and camera position about the area division of a 
picture, a camera angle, a focal distance, image size, tiie position information with time oh 
each terminal, and car information from each tennlnal 10 via the communications networic 30, 
The communication control part 23 which transmits label Information and rally ranking label 
information to the terminal 10, The map information Management Departinent 21 which gains 
the staicture which manages map information, asks for view space in map Infomiation space 
based on the camera position, the camera angle, the focal distance, and image size which 
were received, and exists all over the view space, The game information iVIanagement 
Department 25 which manages the position information with time and car information of each 
tenninal 10. and manages the precedence information in the rally of each car which the user 
who participates In an automobile rally game drives, The label information which matches the 
structure gained to the subregion of said picture, and includes the name or the attribution 
infomiation, and the grant position of a structure which were matched is created, Based on a 
user's position infonrnation and car information under rally game Intervention, When tiie 
registration number of the car has been recognized from the number plate of tiie car and the 
registration number can presume the car which exists In a landscape image to be a registi^tion 
number of the car which a certain user drives from a point of position information and car 
information, It comprises the label infonrnation preparing part 22 which creates the rally ranking 
label infomiation which made the precedence information of ttie rally game superimpose on 
\he field of the car in a landscape image, and the center control section 24 which controls each 
part 21-23, and 25. 
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[0022]Next, operation of this embodiment is explained witli reference to tlie flow cliart of 
drawing 2 . 

[0023]First, in order that the terminal control section 16 may acquire the information about a 
landscape image, a processing start command is sent to the position information acquisition 
part 12, the camera attribution information acquisition part 13, and the image acquiring part 11. 
The position information acquisition part 12 collects position information per second by a GPS 
receiver etc. in response to a command from the terminal control section 16, and passes it to 
the terminal control section 16 (Step 21). Here, a time interval is good not only for how [ a 
second bit but]. The camera attributfon information acquisition part 13 acquires the camera 
angle of landscape image recorders, such as a camera at the time of picture photography, in 
the group of a horizontal angle and an ascending vertical angle in response to the command of 
the terminal control section 16 (Step 22), and if It is a landscape image device which has a 
zoom function simultaneously, it will acquire a focal distance (Step 23). The image acquiring 
part 11 acquires a landscape Image per second from the terminal control section 16 In 
response to a command, and passes it to the control section 16 (Step 24). Since Image size is 
immobilization for every landscape image device, the tenninal control section 16 holds image 
size information. The tenninal control section 16 holds the collected information as a 
landscape image file. 

rQ024 1Drawing 3 s hows the file forniat of tiie data structure of a landscape image file. A 
landscape image file has header information and image data. As header information, it has 
position infonnation, camera angle Infomnation, a focal distance, time information, the image 
size of a graphics file, a type, and size. As position infomnation, it has each data (for example, 
east longitude 137-degree 55 minutes and 10 seconds, north latitude 34-degree 30 seconds of 
34 minutes, altitude of 101 m 33 cm, etc.) of the east longitude, tfie north latitijde, and altitude. 
As a camera angle, it has each data (for example, 254 horizontal-angle right-handed rotations, 
15 ascending vertical angles, etc.) of a horizontal angle and an ascending vertical angle. Focal 
distance data is the focal distances (fbr example, 28 mm etc.) of the camera lens at the time of 
picture photography. As time information, it has the time at the time of photography (fbr 
example, Japan Standard Time 15:06 17 etc. seconds etc. on January 31, 1997). As image 
size of a graphics file, it has pbcel size (for example, 640x480 grade) in every direction. 
Similarly it has file types (TIFE fomn, an 8-bit color, etc.). Similarly ft has the numbers of bytes 
(307,2 KB etc.) of a file. It has the image data itself in a binary format 
[00251lf tiie terminal control section 16 stores a landscape image file, to the image processing 
portion 14, it wilj extinct a border line from a landscape image, and will order to divide a 
landscape image into two or more fields. In the Image processing portion 14, if it says roughly, 
a differential process will be perfonmed based on tiie density difference in a landscape image, 
a bonder line will be extracted (Step 25), and area division is carried out by performing labeling 
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bordering on the border line (Step 26). The technical term called labeling used here is a 
technical term used in the area division of a picture. As a procedure, a picture is first changed 
Into monochrome shade image. Since an outline is a portion into which a luminosity changes 
suddenly, a boreler line is extracted by performing a differential process and asking for a 
portion with a larger differential value than a threshold. At this feme, the line vi^idth of a border 
line Is 1 pixel, and the border line is connected. Therefore, thinning is performed and the 
connected line with a line width of 1 pixel Is obtained. A differential process and thinning are 
enough here, if a certain technique is used from the fomier. 

[0026]The obtained border line is considered to be a border line of a field, and operation of 
numbering the field constituted by a border line is performed. The maximum number in the 
number turns into the number of fields, and the pixel number in a field expresses the area of 
the field. The example which divided the landscape image into two or more subregions is 
shown in dravwng 8 . The measure of the similarily (nearness) between fields may be 
Introduced and clustering processing which summarizes two or more fields In which character 
is alike to one field may be performed. It is good also by the clustering method like the existing 
method throat 

[0027]|f area division processing of a landscape image is completed, the communicatioh 
control part 15 will transmit the information (header infonnation of a landscape image file) 
about the area division of the acquired camera position, a camera angle, a focal distance. 
Image size, and a picture to the center 20 with directioris of the terminal control section 16 
(Step 27). These information Is passed to the map information Management Department 21 of 
the center 20 through the communication control part 23 and the center control section 24, and 
the map infonnation Management Department 21 accesses the map DB, and performs 
calculation processing of view space (Step 28). There is a map database program as an 
example of the map information Management Department 21. The map infonnation 
Management Department 21 has managed three-dimensional map data. Although two- 
dimensional map data may be sufficient, since there is no height information in that case, the 
accuracy of the grant position of ttie labeling to a real landscape is inferior. Height information 
is compensated and processed when canying out based on two-dimensional map data. For 
example, if there is number-of-stories information as which a house expresses several stories 
when it is two dimensional data of a house, three-dimensional data will be created based on 
the height information which multiplied the number of stories by fixed numbers, presumed the 
height of the house, was presumed to be two dimensional data, and was searched for. Even 
when there is no number-of-stories Information, height infonnation can be presumed by 
assigning a fixed number of height according to the area of a house figure, and three- 
dimensional data is similarly created based on presumed height infonnation. In ttils way, three- 
dimensional data Is created and processing is advanced. 
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[0028IThe example of three-dimensional map data is shown in drawing 4. The map Infbmiatfon 
space expressed by two dimensions is shown in drawing 4 and the map information space 
expressed by the three dimension is shown in drawing 4 (2). In response to the command of 
the center control section 24, view space is computed based on the header infomiation of a 
landscape image file to this three-dimensional map information space at the map information 
Management Department 21 (Step 28). The example computation of view space is shown in 
drawing 5. First, XY axis shall stretch horizontally and the Z-axIs shall stretch perpendicularly. 
From the position infonnatlon In the header Information of a landscape image file, the position 
of the viewpoint E Is set up In three-dimensional map information space. For example, if It is 
east longitude 137.degree 55 minutes and 19 seconds, and north latitude 34-degree 30 
seconds of 34 minutes and the altitude of 101 m 33 cm, the corresponding coordinates In the 
map mesh number corresponding to it will be set up. Similarly a camera angular direction is set 
up based on the horizontal angle and ascending vertical angle in the camera angle information 
In header Infomiation. The focus F is taken at the point which progressed by the focal distance 
from the viewpoint E on the straight line showing a camera angular direction. A sight line 
direction vector is a unit vector of the length 1 which comes out from the viewpoint E on the 
straight line. The width x in the X-axis of a camera screen is set up from lateral size with the 
Image size of a landscape image file, and the width y in a Y-axis is set up from the size of a 
lengthwise direction. To a sight line direction vector, the flat surface of the horizontal x length y 
is vertical to a camera angular direction, and it is set up so that the focus F may be included. It 
asks for the straight line which connects the point of four corners of a camera screen from the 
coordinates of the viewpoint E respectively, and let three-dimensional space which four half 
lines prolonged from the viewpoint E make be view space. The example of the view space in 
three-dimensional map space is shown in drawing 6. It looks at three-dimensional map space 
from XZ flat suriiace. The portion sunrounded with the slash in drawing 6 I s a sectional view in 
XZ flat suriace of the space belonging to view space. The building and mountain In view space 
are contained In the example of drawing 6 . 

[0029]lt asks for the structure which exists in the v^ew space for which it asked at the map 
Information Management Department 21. Each peak which constitutes the solid showing a 
structure calculates whether It exists In the interior area of view space for every stnjcture. 
Usually, two-dimensional map space Is divided In the two-dimensional mesh of certain size. In 
addition to the mesh of the direction of two dimensions in every direction, as how to cut the 
mesh of three-dimensional map space, a mesh is cut with the constant interval also to the 
height direction. Space will be divided in the unit space of a rectangular parallelepiped. First, 
the existence of a lapped part with view space is investigated the whole unit space of a 
rectangular parallelepiped, and the number of three-dimensional unit map space with a lapped 
part is searched for. The number of three-dimensional unit map space her© is the same as that 
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what is called of a mesh number. The existence of view space and a lapped part is 
Investigated to the staicture in three-dimensional unit map space with a lap. It asks for the 
straight line which connects the coordinates of the peak and the coordinates of a viewpoint 
which constitute a structure, and if the straight line has an intersection to the camera screen of 
drayy jn fl Z, It is In view space. If at least the one peak fulfills this condition among two or more 
peaks which constitute a structure, that sfructure shall have view space and a lapped part 
[0030IWhen a structure is contained in the Inside of view space or that part is included, the 
processing which cames out three-dimensional projection conversion of each stmcture to this 
surface of projection is started by using a camera screen as a surface of projection (Step 29). 
Here, as shown in drawing 7. after reexpressing the point P by the coondinate system based 
on the viewpoint E based on a following formula (1). the point P is projected on a camera 
screen and it asks for the intersection Q. 
[0031] 
[Equation 1] 




Coordinate-vectors L= of the coordinate-point E=(ex, ey, ez):viewpolnt of the peak which 
constitutes a point P=(x, y, z):structure here (Ix. ly, Iz): Sight line direction vector (unit vector) 
Coordinates r=(lx^+iy^) ""^ intersection Q= at the time of expressing by the coordinate system 
based on the viewpoint E of the point P'=(x'. /. z'):point P (K. Y): The projected point t to the 
camera screen of the point P is a focal distance. [0032]ln three-dimensional projection 
conversion, the fieW which the peak stretches for every structure first Is searched for. For 
example, six fields can be found if It Is a structure expressed In a rectangular parallelepiped. 
When canying out projection conversion of each field to a camera sCTeen, to each pixel on the 
camera screen included in a projection area, the distance of a viewpoint and the corresponding 
points on the field is calculated, and it stores in a memory as a depth value (Z value). For 
every field of each structure, the depth value (2 value) over each pixel on a camera screen is 
calculated, and It stores in a memory, z' in one expresses the depth value (Z value) from a 
viewpoint (fomiula). 

[0033]There are a structure which appears from a viewpoint, and a structure not appearing In 
inside of a structure by which three-dimensional projection conversion was carried out to a 



http-yAvww4.ipd!.mpit.goop/cgi-bin/tranjveb_c^^^ 4/15/^009 

PA(£ 32(51 ' RCVD AT SI13/2009 11:27:09 AM [Eastern Daylight Time]' SVI^USPT^^^^ 



•MAY. 13. 200921 0:30/%TAJDLlA_VTJMORI-nON] NO- ^%cllJ}4 



camera screen. It is necessary to ask only for a structure which appears from a viewpoint In it 
and to search for a field located from a viewpofnt in an opposite hand, and a field internjpted 
by other structures. Then, hidden surface elimination is performed (Step 30). Although it is In a 
method of hidden surfiaoe processing variously, a 2 buffer algorithm is used, for example. 
Other scanline algorithms and the ray tracing method may be sufficient 
[0034]A field which takes a pixel on a camera screen arbitrarily and takes the smallest depth 
value to the pixel is searched for. Thus, if sequential operation is continued about each field of 
each structure, a field nearest to a viewpoint will be left behind for every pixel on a camera 
screen. Since a camera screen top pixel in which a field nearest to a viewpoint is detemiined 
for every pixel on a camera screen, and a field nearest to a viewpoint is common generally 
constitutes a field, in a camera screen, two or more fields which consist of a pixel which makes 
a common field the nearest field are made. In this way, a field which was able to be found is a 
field of a result of having carried out three-dimensional projection conversion of the subreglon 
of a structure which appears from a viewpoint. A field in an opposite hand and a field 
interrupted by other structures are eliminated from a viewpoint 
[0036Jln this way, a made field forms a CG picture area (Step 31 ). 

[0036JTO apex coordinates of a two-dimensional figure which constitutes a CG picture area, 
three-dimensional coordinates before projection conversion are searched for, and It stores in a 
memory by making both conrespondence relation into link infomiation. It is used for asking etc. 
of which structure the two-dimensional field is a projection based on link infomiation. 
[0037JA hidden line removal is canled out and area division of the CG image is carried out 
based on the remaining line data Since the three-dimensional nnap DB is used, a name of a 
structure which sen/es as a basis of the field for every field can be matched. A field to which a 
CG image was divided is numbered in order. An example which divided a CG image Into two 
or more subregions is shown in drawing 9 . 

(0038]If area division processing of a CG Image is completed, the center control section 24 will 
order to the label information preparing part 22 to match a region division of a CG image, and 
a region division of a landscape image. In the label infomiation preparing part 22, a region 
division of a CG image and a region diwsion of a landscape Image are matched by template 
matching (refer to Step 32 and drawing 10) . . 

[0039]lt matches with a region division of a CG image sequentially from a young field (for 
example, No. 1) of a number among region divisions of a landscape image. In matching, 
although it is very good in which [ of the former to a certain MACHINGU methods ], the simple 
template-matching method is taken here. That is, when you pile up two fields to compare and 
there is a ratio of an overlapping portion more than a fixed ratio fixed as a threshold, suppose 
that it matches as a field about the same stnicture. For example, a coordinate value of each 
pixel in the field Is set to (A, B) about R1 of the 1st region division of a landscape image. 
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Values of a pixel In coordinates (A, B) are an Inside of a fiefd. therefore i. m the 1st region 
division SI of a CG image, if coordinates (A, B) become In the field S1. it will be the pixel value 
1 and will lap. but if it becomes outside SI . it will be the pixel value 0 and will not lap. In this 
way. as the lap coefficient K (A. B) in coordinates (A, B). 1 and when lapping, and not lapping, 
It is decided by 0. Coondinates (A. B) are moved in the field R1 , and the lap coefficient K (A. B) 
Is calculated. And to several N1 of coordinates (A. B) moved in the field R1 , the lap coefficient 
K (A. B) calculates several N2 of coordinates which were 1. and a thing corresponding to a 
case where N1/N2 are more than a threshold, in the region dMslon R1 of a landscape image 
and the region division S1 of a CG image is decided on. This matching is performed from the 
1st of a region division of a landscape image to the last thing. Even if a position gap of some is 
In an XY direction in addition to this as a matching method, a valuation function which 
becomes the same value may be used. 

[0040JAfter matching subregion of a CG image to subreglon of a landscape image in the label 
information preparing part 22, It is nnatched, a structure of subreglon Is extracted (Step 33), 
and processing (Step 34) which searches for infomiafion (a name or attribution Infbrmation of a 
structure) which should be further superimposed for every subreglon of a landscape Image, 
and is created as label information with a position which should be superimposed is started. 
Rrst subregion of a corresponding CG Image is taken out to subregion of a landscape image. 
Subregion of a teken-out CG image is obtained by carrying out three-dimensional projection 
conversion of the field which has a three-dimensional structure thing in three-dimensional map 
space from the first to a camera screen. Then, a depth value (Z value) In which subregion of a 
CG image Has a field of a three-dimensional structure thing used as a basis of three- 
dimensional projection conversion is calculated as a key. When three-dimensional projecBon 
conversion is earned out previously, created link information may be used as a key. Based on 
a field of a structure used as a basis, the three-dimensional map DB is accessed and a name 
or attribution infom^aton of the structure Is acquired. It is [anything ] good If attribution 
Infomration Is information which means information which accompanies about the structure 
and starts the structure here. And a position coordinate which should superimpose a name or 
attribution inlbnnation is decided to subregion of a landscape image. How to detemiine may 
determine how. For example, the center of figure which stretches subregion may be sufficient 
Label information is created from a name or attribution Information, and a grant position 
coordinate of the structure. An example of label Information is shown in Table 1 . 
[0041] 

[Table 1] 
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The label informaf on preparing part 22 will pass label Information to the center control section 
24, if It finishes creating label inforination. The center control section 24 will order to transmit 
label information to the terminal 10 to the communications contnal 23. if label Infonnatlon is 
received. (Step 35). 

[0042]Labet infonnation Is passed to the label infomiation outputtlng part 17 of the terminal 10, 
and the label infomiafon outputtlng part 17 superimposes a name or attribution Information of' 
a structure in label Information on a position In a landscape image (Step 36), A landscape 
image on which It was superimposed Is displayed on graphic display devices, such as a 
display (CRT) and a head MARAN toddy spray (Step 37). An example of a landscape image In 
which it was superimposed on label information is shown In drawing 11 . 
[0043]lf an automobile rally game is started between a user of the terminal 10, and a user of 
other temriinals. the communication control part 15 will send position infomiation with time and 
car infomnation of a car which a user of the terminal 10 operates to the center 20 for every 
fixed time with position Information with time on the terminal 10 (Step 38). Through the center 
control section 24. position infomiation with time and car information of a car of the terminal 10 
are passed to the game infomiation Management Department 25 of the center 20, and the 
game infomiation Management Department 25, Each user's position Infonnation with time and 
car Infomiation are managed, precedence Infomiation in a rally of each car which a user who 
participates in an automobile rally game drives Is managed, and a user's position information 
and car information are passed to tfie center control section 24 (Step 39). A user's position 
infomiation and car Infomiation are passed to the label information preparing part 22 by tiie 
center control section 24. and in the label Information preparing part 22. When a registration 
number of the car has been recognized from a number plate of the car, a car which existe in a 
landscape image based on a user's position infomiation and car Information under rafly game 
intervention. If the registration number can presume a registration number of a car which a 
certain user drives from a point of position infomiation and car information, rally ranking label 
infomiation on which a field of a car in a landscape Image was made to superimpose 
precedence information of a rally game will be created, and the center control section 24 will 
be passed (Step 40). 
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[0044]The center control section 24 transmits rally ranking label Infomiation to the terminal 10 
(Step 41). The label information outputting part 17 of the terminal 10 displays rally ranking 
label information on the windshield of a car (Step 42) 
[0045] 

[Effect of the InventionlSlnce according to this invention the geographical information on a 
computer and each portion in the landscape Image of a real landscape can be matched and a 
user can be shown, as explained above. Human being compares the map and real landscape 
on a computer, and it does not match at human beings. - also ends, and an automobile rally 
game is made in the actual street, and the existence and ranking understand only a rally 
participating vehicle In a national highway etc. 



frranslafaon done.] 
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